The Impact of Holistic Review of Urology Residency Applications on Selection for Interview During the COVID-19 Pandemic.

OBJECTIVE: To investigate the impact of a holistic review of urology residency applications on interview selection at our institution during the COVID-19 pandemic. METHODS: In the 2019-2020 cycle, applicants were filtered by a Step 1 score of 230 and whether they applied from selected east coast medical schools. For the 2020-2021 and 2021-2022 cycles, we implemented a scoring system which focused on desirable attributes based on our program training needs and resources. We compared applicant and interviewee demographics and United States Medical Licensing Examination (USMLE) scores using descriptive statistics and 1-way analysis of variance tests. RESULTS: A total of 282, 300, and 367 students applied to our residency program with 50, 45, and 52 selected for interviews during the 2019-2020, 2020-2021, and 2021-2022 cycles, respectively. Compared to 2019-2020, the 2020-2021 and 2021-2022 interviewee cohorts comprised of more non-tri-state applicants (36%, 55.6%, and 46.2%, respectively). Underrepresented minority representation increased for the 2020-2021 interviewee cohort; however, this was not observed in 2021-2022 (16%, 24.4%, 15.4%, respectively). Additionally, USMLE Step 1 and 2 scores were similar between interviewee cohorts in 2019-2020, 2020-2021 and 2021-2022, respectively (Step 1: 244.2 ± 8.8, 242 ± 12.1, 242.8 ± 12.4, P = .624) (Step 2: 249.1 ± 11.5, 251.5 ± 10.5, 254.4 ± 10.8, P = .143). CONCLUSION: Utilizing a comprehensive review resulted in a geographically diverse interview pool and no significant difference in academic performance among interviewees. Holistic review provides an alternative, balanced evaluation of residency applicants which may increase diversity in urology.

Learning From the 2021 Ophthalmology Match: Virtual Residency Matching During the COVID-19 Pandemic.

Background: The effect of virtual interviews on residency match outcomes during the COVID-19 pandemic is unknown. Examining the ophthalmology match may help inform all specialties undergoing virtual interviews. Objective: To determine the impact of allopathic applicant match characteristics in the first year of the virtual residency Match process. Methods: Using the Association of University Professors of Ophthalmology match database, a retrospective review was conducted of all allopathic applicants to ophthalmology residency programs in the United States from the 2016 through the 2021 match cycles. Demographic information, interview numbers, and match outcomes were compared between the 2016-2020 (in-person) and 2021 (virtual) cycles. Results: A total of 3343 allopathic applicants were analyzed. Applicants in the 2021 Match applied to significantly more programs than 2016-2020 applicants did (78.7±23.6 vs 73.1±22.7, P<.001). Among matched and unmatched applicants, there was no significant difference in the number of interviews granted or completed. There was a significant reduction in the match rate between the 2016-2020 and 2021 Match cycles (81.3% vs 76.6%, P=.0009). A subanalysis of applicants who went to medical schools with ophthalmology residency programs (N=2308) found that the home institution match rate was significantly higher for the 2021 Match compared to the aggregate 2016-2020 Matches (26.1% vs 20.6%, respectively, P=.015). Conclusions: Significantly more applicants to ophthalmology residency programs matched at their home institutions in the 2021 virtual match cycle compared to the previous 5 years without influencing the interview numbers granted and attended.

Inducting undergraduate medical students via WhatsApp-based multiple mini interviews during COVID-19 pandemic.

Selection of medical students requires both cognitive and soft skills assessment. Shalamar Medical and Dental College (SMDC) has been using on-campus multiple mini interviews to assess the latter but due to Covid-19 pandemic it became imperative that an alternative be found. The aim of this communication is to share the process SMDC went through to plan, design, and ultimately conduct WhatsApp-based multiple mini interviews (wMMI) in a low risk method as an entry criteria for undergraduate medical students. The process involved designing scenarios appropriate for online interviews, training the faculty members regarding conducting MMI as well as the use of technology, and designing an online webpage for enrolling, scheduling and assessing candidates. We were able to successfully complete wMMI process for 522 candidates within one week in a low risk setting using WhatsApp as communication medium with strong IT and administrative support.

Urology Residency Applications in the COVID-19 Era.

OBJECTIVE: To evaluate urology applicants' opinions about the interview process during the COVID-19 pandemic. MATERIAL AND METHODS: An anonymous survey was emailed to applicants to our institution from the 2019 and 2020 urology matches prior to issuance of professional organization guidelines. The survey inquired about attitudes toward the residency interview process in the era of COVID-19 and which interview elements could be replicated virtually. Descriptive statistics were utilized. RESULTS: Eighty percent of urology applicants from the 2019 and 2020 matches received our survey. One hundred fifty-six people (24% of recipients) responded. Thirty-four percent preferred virtual interviews, while 41% in-person interviews at each program, and 25% regional/centralized interviews. Sixty-four percent said that interactions with residents (pre/postinterview social and informal time) were the most important interview day component and 81% said it could not be replicated virtually. Conversely, 81% believed faculty interviews could be replicated virtually. Eighty-seven percent believed that city visits could not be accomplished virtually. A plurality felt that away rotations and second-looks should be allowed (both 45%). COMMENT: Applicants feel that faculty interviews can be replicated virtually, while resident interactions cannot. Steps such as a low-stakes second looks after programs submit rank lists (potentially extending this window) and small virtual encounters with residents could ease applicant concerns. CONCLUSION: Applicants have concerns about changes to the match processes. Programs can adopt virtual best practices to address these issues.

Increasing access to the profession: Admissions lessons learned from the pandemic.

INTRODUCTION: The disruptions induced by the COVID-19 pandemic have forced quick and significant changes to recruitment and admissions practices in colleges of pharmacy. This process has helped to identify barriers and challenges for prospective students. At the University of North Carolina Eshelman School of Pharmacy, some changes were already under consideration prior to the pandemic, such as moving to test-optional admissions and allowing remote interviews, while new considerations included offering the entire recruitment and admissions process remotely. METHODS: In 2020-2021, the decision was made to move to test-optional admissions. A separate decision was made to conduct interviews remotely. Data from the admission cycle were collected from the Pharmacy College Application Service as part of the standard admissions process and exported for analysis. Descriptive statistics (mean ±â€¯SD) were used. RESULTS: Completed applications increased by 59.1% in 2020-2021 from the previous year. Applications increased by 9.8% from underrepresented students, by 6.2% from those with a bachelor's degree, and by 8.4% by out of state students. Other admissions metrics, such as the mean grade point average (3.50) and mean Pharmacy College Admissions Test composite percentile (88%), did not change. CONCLUSIONS: The COVID-19 pandemic experience validated our perspective that we must continue to embrace change and seize opportunities to reduce barriers for prospective students to improve access to the profession. The changes that this pandemic has necessitated may help to close the gaps in accessing health professions education.

UCAT and dental student selection in the UK - what has changed?

Introduction The University Clinical Aptitude Test (UCAT) has been used since 2006 by a consortium of UK medical and dental schools to assist in undergraduate selection. In 2019, UCAT was used by 30 universities (14 dental schools).Aim To report how UCAT use has changed in undergraduate student selection in the UK.Methods UCAT use was categorised and trends identified from annual telephone interviews with dental school admission tutors; this process started in 2011.Results Dental schools using UCAT rose from 8 (2006) to 14 (2020). The most significant use of the test to select applicants for interview was as a weighted factor; at offer stage, UCAT was most used to discriminate between applicants at borderlines. A growing number of dental schools are using the Situational Judgement Test (SJT) in selection (2019, n = 6). In 2019, eight schools adjusted selection processes for widening access applicants. Multiple mini interviews are now used by the majority (n = 10) of dental schools.Conclusions UCAT represents a significant factor in selection to UK undergraduate dental programmes and is used by all but two dental schools. In most schools, UCAT contributes in a substantial way to selection outcomes and strength in test use has grown over time.

Feasibility of an automated interview grounded in multiple mini interview (MMI) methodology for selection into the health professions: an international multimethod evaluation.

OBJECTIVES: Global, COVID-driven restrictions around face-to-face interviews for healthcare student selection have forced admission staff to rapidly adopt adapted online systems before supporting evidence is available. We have developed, what we believe is, the first automated interview grounded in multiple mini-interview (MMI) methodology. This study aimed to explore test-retest reliability, acceptability and usability of the system. DESIGN, SETTING AND PARTICIPANTS: Multimethod feasibility study in Physician Associate programmes from two UK and one US university during 2019-2020. PRIMARY, SECONDARY OUTCOMES: Feasibility measures (test-retest reliability, acceptability and usability) were assessed using intraclass correlation (ICC), descriptive statistics, thematic and content analysis. METHODS: Volunteers took (T1), then repeated (T2), the automated MMI, with a 7-day interval (±2) then completed an evaluation questionnaire. Admission staff participated in focus group discussions. RESULTS: Sixty-two students and seven admission staff participated; 34 students and 4 staff from UK and 28 students and 3 staff from US universities. Good-excellent test-retest reliability was observed at two sites (US and UK2) with T1 and T2 ICC between 0.65 and 0.81 (p<0.001) when assessed by individual total scores (range 80.6-119), station total scores 0.6-0.91, p<0.005 and individual site (≥0.79 p<0.001). Mean test re-test ICC across all three sites was 0.82 p<0.001 (95% CI 0.7 to 0.9). Admission staff reported potential to reduce resource costs and bias through a more objective screening tool for preselection or to replace some MMI stations in a 'hybrid model'. Maintaining human interaction through 'touch points' was considered essential. Users positively evaluated the system, stating it was intuitive with an accessible interface. Concepts chosen for dynamic probing needed to be appropriately tailored. CONCLUSION: These preliminary findings suggest that the system is reliable, generating consistent scores for candidates and is acceptable to end users provided human touchpoints are maintained. Thus, there is evidence for the potential of such an automated system to augment healthcare student selection.

Defying the Pandemic, Applications to Nursing Schools Increase.

But faculty shortages and other challenges continue to limit enrollment.

The current dental school applicant: an overview of the admission process for UK dental schools and the sociodemographic status of applicants.

Aims To determine the current processes used to assess dental school admissions in the UK as well as compare the applicants' demography.Methods All 16 dental schools in the UK were invited to complete a questionnaire analysing the admissions protocols between 2018-2019 and 2019-2020. These data were combined with the admission process information available online. Retrospective data from the University Clinical Aptitude Test including the sociodemographic status of dental applicants were collected. For the six dental schools that did not reply, data presented were collected from information available to the public.Results The majority of applicants were women (63.2% in 2019-2020), white (27.9%) and are sixth form attendees or attend a further education college (40.6%). Of those who apply to study dentistry, 15.5% are graduates who hold a first degree. For each undergraduate dental place available, there were 12.6 applicants and 9.4 applicants to each post-qualified dental undergraduate place.Conclusion Further advancements are required to widen participation and broaden the sociodemographic status of dental applicants. This area would benefit from a long-term prospective study about recruitment methods and its correlation with performance at dental school. COVID-19 is impacting the application process, the full extent of which is yet to be determined.

Predictive validity of A-level grades and teacher-predicted grades in UK medical school applicants: a retrospective analysis of administrative data in a time of COVID-19.

OBJECTIVES: To compare in UK medical students the predictive validity of attained A-level grades and teacher-predicted A levels for undergraduate and postgraduate outcomes. Teacher-predicted A-level grades are a plausible proxy for the teacher-estimated grades that replaced UK examinations in 2020 as a result of the COVID-19 pandemic. The study also models the likely future consequences for UK medical schools of replacing public A-level examination grades with teacher-predicted grades. DESIGN: Longitudinal observational study using UK Medical Education Database data. SETTING: UK medical education and training. PARTICIPANTS: Dataset 1: 81 202 medical school applicants in 2010-2018 with predicted and attained A-level grades. Dataset 2: 22 150 18-year-old medical school applicants in 2010-2014 with predicted and attained A-level grades, of whom 12 600 had medical school assessment outcomes and 1340 had postgraduate outcomes available. OUTCOME MEASURES: Undergraduate and postgraduate medical examination results in relation to attained and teacher-predicted A-level results. RESULTS: Dataset 1: teacher-predicted grades were accurate for 48.8% of A levels, overpredicted in 44.7% of cases and underpredicted in 6.5% of cases. Dataset 2: undergraduate and postgraduate outcomes correlated significantly better with attained than with teacher-predicted A-level grades. Modelling suggests that using teacher-estimated grades instead of attained grades will mean that 2020 entrants are more likely to underattain compared with previous years, 13% more gaining the equivalent of the lowest performance decile and 16% fewer reaching the equivalent of the current top decile, with knock-on effects for postgraduate training. CONCLUSIONS: The replacement of attained A-level examination grades with teacher-estimated grades as a result of the COVID-19 pandemic may result in 2020 medical school entrants having somewhat lower academic performance compared with previous years. Medical schools may need to consider additional teaching for entrants who are struggling or who might need extra support for missed aspects of A-level teaching.

Conducting multiple mini-interviews in the midst of COVID-19 pandemic.

Multiple mini-interview (MMI) is a 'multiple sample-based' approach comprising multiple focused encounters intended to access and assess a range of attributes in order to gain more objectively multiple impressions of an applicant's interpersonal skills, thoughtfulness and general demeanour. It is designed to focus on four domains that are not considered to be comprehensive, but are considered to be vital for a successful career in the health sciences: critical thinking, ethical decision making, communication and knowledge of the healthcare system. Traditionally, the MMI is conducted face-to-face, but with COVID-19 pandemic and the implementation of social distancing measures, no onsite or campus teaching, banning of mass gatherings and cancellation of face-to-face interviews, Pengiran Anak Puteri Rashidah Sa'adatul Bolkiah Institute of Health Sciences at Universiti Brunei Darussalam explored the feasibility of conducting MMI through virtual means. This report provides an account of our experience in conducting internet-MMI for the selection of new applicants into the August 2020 cohort of the Medicine programme. We also aimed to determine whether the scores derived from internet-MMI were reliable and equivalent to the scores derived from traditional MMI.

Virtual interviews - Utilizing technological affordances as a predictor of applicant confidence.

PURPOSE: In the midst of a pandemic, residency interviews transitioned to a virtual format for the first time. Little is known about the effect this will have on the match process. The study aim is to evaluate resident application processes and perceived outcomes. METHODS: An electronic survey was distributed to 142 colon and rectal surgery residency applicants (95% of total). RESULTS: A total of 77 applicants responded to the survey (54% response rate). Applicants reported high levels of satisfaction with virtual interviews but less comfort. Utilizing the mute button and using notes in a different way from face-to-face interviews were significantly associated with applicant confidence that they ranked the right program highest. A majority of applicants (73%) would recommend virtual interviews next year even if COVID-19 is not a factor. CONCLUSION: While applicants appear generally satisfied with virtual interviews, they also reported less comfort. Applicant confidence was predicted by utilizing the unique technological affordances offered by the virtual platform.

Virtual Residency Interview Experience: The Child Neurology Residency Program Perspective.

BACKGROUND: The COVID-19 pandemic presented many challenges for graduate medical education, including the need to quickly implement virtual residency interviews. We investigated how different programs approached these challenges to determine best practices. METHODS: Surveys to solicit perspectives of program directors, program coordinators, and chief residents regarding virtual interviews were designed through an iterative process by two child neurology residency program directors. Surveys were distributed by email in May 2021. Results were summarized using descriptive statistics. RESULTS: Responses were received from 35 program directors and 34 program coordinators from 76 programs contacted. Compared with the 2019-2020 recruitment season, in 2020-2021, 14 of 35 programs received >10% more applications and most programs interviewed ≥12 applicants per position. Interview days were typically five to six hours long and were often coordinated with pediatrics interviews. Most programs (13/15) utilized virtual social events with residents, but these often did not allow residents to provide quality feedback about applicants. Program directors could adequately assess most applicant qualities but felt that virtual interviews limited their ability to assess applicants' interpersonal communication skills and to showcase special features of their programs. Most respondents felt that a combination of virtual and in-person interviewing should be utilized in the future. CONCLUSIONS: Residency program directors perceived some negative impacts of virtual interviewing on their recruitment efforts but in general felt that virtual interviews adequately replaced in-person interviews for assessing applicants. Most programs felt that virtual interviewing should be utilized in the future.

Perceptions of the Virtual Neurosurgery Application Cycle During the Coronavirus Disease 2019 (COVID-19) Pandemic: A Program Director Survey.

OBJECTIVE: The novel coronavirus disease 2019 (COVID-19) pandemic has led to a shift to virtual residency interviews for the 2020-2021 neurosurgery match, with unknown implications for stakeholders. This study seeks to analyze the perceptions of residency program directors (PDs) and associate program directors (APDs) regarding the current virtual format used for residency selection and interviews. METHODS: An anonymous, 30-question survey was constructed and sent to 115 neurosurgery PDs and 26 APDs to assess respondent demographics, factors used to review applicants, perceptions of applicants and applicant engagement, perceptions of standardized letters and interview questions, the effect of the virtual interview format on various stakeholders, and the future outlook for the virtual residency interview format. RESULTS: A total of 38 PDs and APDs completed this survey, constituting a response rate of 27.0%. Survey respondents received significantly more Electronic Residency Application Service applications in the 2020-2021 cycle compared with the 2019-2020 cycle (P = 0.0029). Subinternship performance by home-rotators, (26.3%), letters of recommendation (23.7%), and Step 1 score (18.4%) were ranked as the most important factors for evaluating candidates during the current virtual application cycle. CONCLUSIONS: Our study highlights that applicants applied to a greater number of residency programs compared with years prior, that the criteria used by PDs/APDs to evaluate applicants remained largely consistent compared to previous years, and that the virtual residency interview format may disproportionately disadvantage Doctor of Osteopathic medicine and international medical graduate applicants. Further exploring attitudes toward signaling mechanisms and standardized letters may serve to inform changes to future neurosurgery match cycles.

Implementation and evaluation of virtual multiple mini interviews as a selection tool for entry into paediatric postgraduate training: A Queensland experience.

INTRODUCTION: The Queensland Basic Paediatric Training Network (QBPTN) is the centralised pathway for entry into paediatric training in Queensland, Australia. In response to COVID-19 travel and social distancing restrictions imposed in 2020, QBPTN successfully adopted a Virtual Multiple Mini Interviews (vMMIs) model for the selection of candidates for entry into paediatric training. The authors describe the planning, implementation, challenges, and evaluation of candidates' and interviewers' experiences of vMMIs, including the differences between candidates from two geographical areas. METHODS: The contents of six vMMI stations were similar to face-to-face MMI. Implementation required the identification of ZOOMTM as a preferred online platform, securing venues, communication, development of contingency plans and central coordination by the network. Candidates' experiences with vMMI were explored through thematic analysis of the qualitative data from focus groups and free text responses, and descriptive analysis of SurveyMonkey© questionnaire responses. Experiences between 'metropolitan' and 'regional and interstate' candidates were compared. RESULTS: 5-minute stations with 2-minute pre-reading were used. 78 candidates and 14 interviewers participated in the selection process. All candidates attended the focus group. 58.7% of candidates responded to post vMMI questionnaire. 93% of survey responders were happy to undertake vMMI in the future, with 23% feeling they would have performed better in face-to-face. Experiences between 'metropolitan' and 'other' groups were similar. Positive experiences of participants were related to the user-friendly IT platform, successful pre-interview communications, preparation, convenience, time, and cost savings. Stress related IT failures and difficulties establishing rapport with interviewers were reported as the main negative experiences. CONCLUSION: 'vMMI' is a feasible and acceptable method of selection into paediatric training. vMMI has many benefits and can be implemented relatively quickly by addressing key logistical requirements. The model under discussion could be adapted by other centres based on local needs.